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Targeting of Cargo Containers 2008: Review of CBP 's Cargo 
Enforcement Reporting and Tracking System (OIG-08-65) 



As required by Section 809(g) of the Coast Guard and Maritime Transportation Act of 2004 
(Public Law 108-293), we conducted an annual audit of the Automated Targeting System (ATS). 
We performed the audit at CBP Headquarters in Washington, D.C. between April 2007 to 
June 2007 under the authority of the Inspector General Act of 1978, as amended, and according 
to generally accepted government audit standards. The objective of our audit was to determine 
whether U.S. Customs and Border Protection (CBP) is effectively developing and implementing 
the Cargo Enforcement Reporting and Tracking System (CERTS). CERTS is designed to gather 
data on cargo examination findings and report on how efficiently examination equipment is 
being used. Analysis of this information can be used to improve CBP's ability to identify high- 
risk cargo containers entering the United States and examination methodologies. 

Results of Audit 

CBP could improve its management and oversight of the development and implementation of 
CERTS. Specifically, CBP has not updated the CERTS project plan, to include the scope of 
work, a detail implementation schedule for system design, development, and testing, and cost 
estimates past Phase 1 . In addition, CBP bypassed key Customs Standard Life Cycle reviews 
designed to ensure end-users have a properly working system and have received management's 
approval to continue the project. As a result, CERTS project development was delayed and not 
fully available to end-users as CBP originally planned. CBP concurred with our 
recommendations and has taken or said it will take prompt steps to develop, implement, and 
monitor an updated CERTS project plan. 
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Background 



The cornerstone for all of CBP's targeting efforts is ATS, an intranet-based enforcement and 
decision support tool to help CBP Officers safeguard our borders by identifying cargo and 
personnel entering the U.S. that need further scrutiny. ATS is comprised of several targeting 
modules, including ATS4-N, used for targeting inbound cargo. CERTS is a subsystem of the 
ATS4-N module. CERTS is being developed to provide a user-friendly, single-point-of-entry for 
CBP Officers to electronically document cargo examination results and provide management 
with information on how efficiently examination equipment is being used. According to CBP, 
CERTS will improve CBP's ability to target cargo for inspections. 

Information system development projects, such as CERTS, require effective management and 
oversight to ensure that they are developed on schedule and within budget, and produce the 
expected results. OMB Circular A-130 requires agencies to use a performance-based 
management system that provides timely information regarding the agency's progress in 
developing and implementing information technology (IT) investments. According to 
Circular A-130, the performance system used by project managers must measure progress 
towards milestones in an independently verifiable basis, in terms of cost, and capability of the 
investment to meet specified requirements, timeliness, and quality. 

CBP's Systems Development Life Cycle Handbook (Handbook), (CIS HB 5500-07A, dated 
February 2, 2001 and revised December 9, 2005 and November 9, 2006), establishes CBP's 
policy on the development of life cycles, processes, and documentation requirements for IT 
projects. The Handbook requires that a project plan be created and reviewed as early in the life 
cycle as possible and later modified, revised, reviewed, and re-approved as more detail is 
developed during the course of the project or as changes are required and corrective actions 
taken. Documenting changes is a mandatory management support activity that occurs at every 
stage in the life cycle. According to the Handbook, all IT projects, pilots, and prototypes must 
follow the Customs Standard Life Cycle (CSLC). The CSLC framework describes the critical 
activities, reviews, and deliverables organized into stages that must occur during a development 
life cycle (See Appendix A for an overview of the stages in the CSLC). The Handbook also 
requires the development of detailed work schedules and estimates that include work products, 
efforts, and costs. 

CBP prepared five iterative versions of the CERTS Project Plan. The first four versions called 
for all CERTS capabilities to be delivered in one release. CBP originally planned to conduct a 
pilot program from March to September 2006 and deploy CERTS to all ports by November 
2006. However, during system development, CBP experienced technical challenges. In 
response, the latest, and fifth, version of the CERTS Project Plan (Version 1.5, dated September 
6, 2006), introduced a phased implementation strategy. The phased implementation strategy 
requires that for each phase of the project, CBP define the scope of work and schedule for system 
design, development, and testing. For Phase 1, CBP selected to implement 40 of the 147 user 
and functional requirements at 22 major seaports. 
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Project Plan Not Fully Revised 

Although the Project Plan was revised to show a phased implementation, the Plan's scope of 
work, schedule, and cost estimates for the CERTS design, development, and testing was not 
updated to coincide with the phased implementation strategy. For the scope of work, CBP did 
not describe the number of CERTS phases, the user and functional requirements to be included 
in each phase, the work to be performed in each phase, or the detailed development methods and 
management strategies as required by the Handbook. 

CBP's schedule for CERTS was not revised and updated to provide for the development and 
implementation of multiple CERTS phases. Consequently, the schedule does not include 
timeframes for the end of the Phase 1 pilot program and the deployment to all subsequent ports. 
In addition, the schedule does not include expected dates for the mandatory CSLC milestones for 
the remaining CERTS phases. 

CBP has not revised, updated, and documented the project plan for cost estimates or sources of 
funding to complete CERTS under the phased implementation strategy. CBP originally 
estimated the cost of CERTS to be $7.2 million over five years, including operations and 
maintenance costs. According to the CERTS budget, CBP will have spent approximately $5.66 
million for FYs 2005 through 2007 and spend the remaining $1.54 million for FYs 2008 through 
2010. This remaining amount does not appear sufficient to complete the CERTS project, based 
on the additional requirements planned with needed stages of development, construction, testing, 
piloting, and deployment. CBP officials said they were aware that additional funding will be 
needed to design, construct, test, pilot, and deploy the remaining phases. 

CBP officials said that delays in updating the project plan and schedule were partly the result of 
technical challenges identified during the pilot program and the need to complete prioritizing the 
remaining CERTS requirements. These technical challenges included complexities in migrating 
to a new ATS architecture, significant performance issues identified during the CERTS testing, 
and problems with developing the software. Updating the project plan will help CBP 
management track and monitor the project and take timely corrective actions to ensure that 
CERTS is developed on schedule and within budget, and produces the expected results. 

Management Reviews By-Passed 

CBP bypassed two significant stages: the Test Readiness Review and the Quality Assurance 
Reviews for Phase 1 of CERTS. CBP may have mitigated some of the technical challenges 
encountered with the software during the Phase 1 pilot program had it conducted those reviews 
at the end of the construction stage. These reviews are designed to present a work product to 
end-users and receive management's approval for continuing to the next phase. 

Test Readiness Review 

CBP bypassed the Test Readiness Review at the end of the construction stage and continued to 
the next stage of development. This review is a formal management meeting intended to ensure 
the proper operation and integration of equipment and software before continuing with the 
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project. Unit and integration testing are key activities where testers and senior managers 
document their concurrence that the system and software are ready to enter the next stage, which 
is the acceptance stage. 

Quality Assurance Review 

CBP bypassed the Quality Assurance Review at the end of the construction stage, and proceeded 
directly into the acceptance stage, where CBP conducted the Phase 1 pilot program. Quality 
Assurance Reviews ensure that user requirements are met, processes are documented and 
followed, work products and activities comply with all applicable standards and requirements, 
and new systems work with existing systems. Quality Assurance teams conduct reviews and 
provide project staff and senior managers with the results. The Handbook requires that quality 
assurance occur at each stage of the CSLC and that a formal Quality Assurance Review occur 
before transition to the acceptance stage. According to CERTS project officials, CBP bypassed 
the Test Readiness Review and Quality Assurance Review to provide CERTS to the end users 
more quickly. CBP used the Phase 1 pilot program to identify and correct application, 
programmatic, functionality, and usability issues. 

Recommendations 

We recommend that the Commissioner, U.S. Customs and Border Protection: 

1. Develop, implement, and monitor an updated CERTS project plan that includes: 

a. Details of the work to be performed in each phase of the project; 

b. Revised schedules for the design, development, testing, and deployment of all 
CERTS phases; and 

c. Cost estimates and sources of funding to complete all CERTS phases. 

2. Utilize the CSLC for all CERTS phases to focus on satisfying user requirements, 
including mandatory reviews to improve management's oversight of the project. 

Management Comments and OIG Analysis 

CBP provided formal comments on a draft of our report (see Appendix B). CBP concurred with 
our recommendations and has taken or will take prompt steps to develop, implement, and 
monitor an updated CERTS project plan. CBP also said it will use the CSLC for all CERTS 
phases to focus on satisfying user requirements, including mandatory reviews to improve 
management's oversight of the project. We agree with the steps CBP has taken, and plans to 
take, and believe these steps satisfy our report recommendations. 

We look forward to learning more about CBP's continued progress in the development and 
implementation of CERTS. Should you have any questions or concerns, please call me, or your 
staff may contact Anne L. Richards, Assistant Inspector General for Audits, at 202-254-4100. 
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Appendix A 

Customs Standard Life Cycle - Overview 



Chapter 4 Customs Standard Life Cycle 
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Figure 4.2, Customs Standard Life Cycle - Overview 
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Customs Standard Life Cycle - Overview 
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Stage 1 - Concept and Business Case: This life cycle stage begins with the business 
need for a particular automated solution and ends with the release of funds to begin 
development. Included in this stage are Enterprise Architecture reviews by TAG and 
reviews by the TRC, ITC, and IRB, depending on project size, cost, risk and funding 
availability, 



X 



Stage 2 - Initiation and Authorization: This stage begins when funding is released to 
begin detailed project planning. This includes developing a draft of user requirements and 
infrastructure planning, a technical compliance review, an initial project plan, and 
appropriate life cycle tailoring. 



Stage 3 -- Project Definition: During Project Definition, information created in the 
previous stage is further refined until a clear set of functional requirements can be 
produced and certified by the Business Sponsor. Based on these requirements, the 
technical planning and support area documentation are created. 



X 



Stage 4 — System Design: During System design, the development team uses the 
requirements to create the technical design of the system under development. This stage 
ends with a Critical Design Review, in which the Business Sponsor reviews the work 
products and certifies that the system design meets the business need. 



X 



Stage 5 — Construction: During this stage, the requirements and design developed 
during the previous stages are translated into operational work products. These work 
products and code then undergo unit and integration testing by the development team and 
supporting organizations until the system is ready for acceptance testing. Also, equipment 
is received and checked to ensure proper operation and integration. 



X 



Stage 6 - Acceptance: There are two types of acceptance that occur during this stage. 
First is system testing to ensure that it interfaces properly with other systems in the 
Customs environment. Second, independent testers and the users test the system to 
ensure that it meets the needs stated in the User Requirements. Security Certification and 
Accreditation are obtained during this stage. 



Stage 7 ~ Operational Readiness: The new system is moved into the Customs 
Production environment, but it is not ready for operational processing. This stage focuses 
on preparations for rolling out the system to all field users, field testing, and finalizing user 
documentation. 



Stage 8 ~ Operations: The system is in general use throughout the Customs Service. 
This stage consists of activitivating and rolling out the system, plus activities to support 
and monitor performance and to ensure continuity of operations. Also reviews are 
performe d to as sess the effectiveness and cost benefits achieved by the investment. 



X 



Stage 9 -- Retirement: When a system no longer meets Customs needs, its existence is 
terminated and any data or modules not needed by replacement systems are archived and 
stored in a secure off -site location. 



Figure 4.3, Customs Standard Life Cycle - Stage Descriptions 
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Appendix B 

Management Comments 



U.S. Department of Homeland Security 

Washington, DC 20229 




5J U.S. Customs and 
m Border Protection 



March 19, 2008 



MEMORANDUM FOR ANNE RICHARDS 

ASSISTANT INSPECTOR GENERAL FOR AUDITS 



Thank you for providing us with a copy of your draft report entitled "Targeting of Cargo 
Containers 2008. Review of the Cargo Enforcement Reporting Tracking System' 1 and the 
opportunity to discuss the issues in this report. 

The U.S. Customs and Border Protection (CBP) agrees with the Department of Homeland 
Security (DHS). Office of Inspector General's (OIG's) overall observations that CBP could 
improve its management and oversight of the development and implementation of the Cargo 
Enforcement Reporting T racking System (CERTS). 

CBP concurs with the two recommendations and has taken or will take prompt steps to 
address develop, implement, and monitor an updated CERTS project plan and utilize the 
Customs Standard Life Cycle (CSLC) for all CERTS phases to focus on satisfying user 
requirements, including mandatory reviews to improve management's oversight of the 
project. CBP will continue to improve the management oversight effectiveness to improve 
customer satisfaction and product quality. 

Attached arc comments specific to the recommendations that relate to statements that need to 
be clarified prior to the finaiization of this report, 

CBP has determined that the information in the audit does not warrant protection and may be 
disclosed to the public. 

If you have any questions regarding this response, please have a member of your staff contact 
Ms. Arlenc Lugo at (202) 344-1218 

Attachments 



FROM: 



Director f >> JL. /</ 

Office of Policy and Planning 

U.S. Customs and Border Protection 



SUBJECT: 



Responses to the Office of Inspector General's Draft Report 
entitled "Targeting of Cargo Containers 20()fj : Review of the 
Cargo Enforcement Reporting and Tracking System" 
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Appendix B 

Management Comments 



2 

CBP Response to OIG Draft Report Entitled "Targeting of Cargo Containers 2008: 
Review of the Cargo Enforcement Reporting and Tracking System" 



Recommendation 


Corrective Action Plan 


Completion 
Date 


Comments 


1. Develop, 
implement, and 
monitor an 
updated CERTS 
project plan 


Realigned with the CBP CSLC. 
Update CERTS CSLC 
documentation. 
Conducted Test Readiness 
Review (TRR) (09/13/07), 
Production Readiness Review 
(PRR) (09/26/07), Operational 
Readiness Review (ORR) 
(12/12/07) 

Conducted latest Quality 
Assurance (QA) review 
(01/1 1/08) 


09/13/2007 


On 02/2672008 a memorandum was issued to the 
field users mandating the use of CERTS Phase 1 
for the Air/Sea environments effective 
March 1, 2008. 


a. Details of 
the work to be 
performed in 
each phase of 
the project 


Maintain a current Work 
Breakdown Structure (WBS). 
Update the Project plan to 
include the certified prioritized 
requirements for CERTS Phase 
II submitted to OIT on 
02/22/200S. 


02/29/2008 


A prioritized list of Phase JI requirements has 
been Identified and approved for Phase [1 
kickoff. Requirements prioritization is ongoing 
for future development efforts. 


b. Revised 
schedules for 
the design, 
development, 
testing, and 
deployment of 
all CERTS 
phases 


Continually Monitor, update 
and manage project 
requirements to mitigate risk 
and provide improved 
Management Oversight. 


Ongoing 


CERTS phases are being conducted in an 
iterative approach. 

All development products are subjected to 
stringent Unit testing, System Integration 
Testing (SIT) and System Acceptance Testing 
(SAT). 

Each iterative development drop is evaluated 
and controlled through OIT's Configuration 
Control Board (CCB) before implementing to 
production. 

As each phase develops, it will be subject to the 
CBP CSLC to include User Acceptance Testing 
(UAT), TRR, PRR, ORR milestones and QA 
reviews. These milestones and detailed work 
will be monitored via the Targeting & Analysis 
Systems Program Office (TASPO) project WBS 
and Worklenz. 


c. Cost 
estimates and 
sources of 
funding to 
complete all 
CERTS 
phases. 


Secure funding to complete 
CERTS development. 
Submit cost estimates when 
appropriate. 


Ongoing 


Remaining CERTS funding was re-appropriated 
by DHS and remains an open issue. 
Funding sources arc under investigation. An 
unfunded request has been submitted for FY08 
funds. 


2. Utilize the 
CSLC for all 
CERTS phases to 
focus on satisfying 
user 

requirements, 
including 
mandatory 
reviews to 
improve 
management's 
oversight of the 
project. 


Utilize CBP'sCLSC 
Framework to Improve 
Management Oversight of the 
project while increasing 
Customer satisfaction. 
Conduct User and Project 
Owner Reviews on a regular 
schedule. 


Ongoing 

since 
12/2007 


On 02/26/2008 a memorandum was issued to the 
field users mandating the nationwide use of 
CERTS Phase I for the Air/Sea environments 
effective March I, 2008. 
TASPO conducts biweekly production 
conference calls with stakeholders to review 
product performance and functionality. 
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Appendix C 
Report Distribution 



Department of Homeland Security 

Secretary 
Deputy Secretary 
Chief of Staff 
Deputy Chief of Staff 
General Counsel 
Executive Secretary 
GAO/OIG Liaison Office 
CBP Commissioner 
CBP Audit Liaison 

Assistant Secretary for Legislative Affairs 
Assistant Secretary for Policy 
Assistant Secretary for Public Affairs 

Office of Management and Budget 

Chief, Homeland Security Branch 
DHS OIG Budget Examiner 

Congress 

Congressional Oversight and Appropriations Committees, as 
appropriate 
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Additional Information and Copies 

To obtain additional copies of this report, call the Office of Inspector General 
(OIG) at (202) 254-4199, fax your request to (202) 254-4305, or visit the OIG web 
site at www.dhs.gov/oig. 



OIG Hotline 

To report alleged fraud, waste, abuse or mismanagement, or any other kind of 
criminal or noncriminal misconduct relative to department programs or 
operations: 

• Call our Hotline at 1-800-323-8603; 

• Fax the complaint directly to us at (202) 254-4292; 

• Email us at DHSOIGHOTLINE@dhs.gov; or 

• Write to us at: 

DHS Office of Inspector General/MAIL STOP 2600, Attention: 
Office of Investigations - Hotline, 245 Murray Drive, SW, Building 410, 
Washington, DC 20528. 

The OIG seeks to protect the identity of each writer and caller. 



